INTRODUCTION

A permanent test organisation is another way to organise the test process than discussed so far. The differences are discussed in the next sections. The services that a test organisation can offer and the way in which agreements on these services are reached are discussed in section Supply Test Services (SP). Section General Process Model then describes a general process model, after which this is elaborated on for two common types of permanent test organisations. These two types are described in section Two Common Types Of Test Organisation. Guideline Role Of A Permanent Test Organisation In Outsourcing explains the role of the permanent test organisation in the phenomenon of outsourcing (of the entire IT organisation or just the testing team).

PERMANENT TEST ORGANISATION EXPLAINED

Organisations can decide to set up a permanent test organisation that offers various test services. A permanent test organisation, contrary to project-based testing, does not implement a specific element of the test process on a per project basis, but across all projects. This does not mean that no test activities are executed within the projects.

There is no set list of activities that can be executed by the permanent test organisation, but this varies per organisation. For instance, the entire preparation and execution of tests can be moved to a permanent test organisation, but also setting up and maintaining the test environment, or setting up and maintaining test tools. As a result, every process in relation to that specific part of testing (test execution, test environment, test tools, etc.) is executed according to a fixed method and with reusable resources. The process will have the same quality regardless of the project. The permanent test organisation provides these elements to projects via so-called test services.

It depends on the client how the elements of the test process are elaborated on by the permanent test organisation (and therefore which test services it delivers). Test services are acquired by clients of the permanent test organisation. The wish of the client determines which services are offered. The client can have many forms, which also determines the service delivered. For instance, when a test environment is delivered, the client may be the specific tester in a project, but also the developer of the application who wishes to try out something. Another example is a test automation service. This, too, may have different clients. It may be the tester who wants an environment in which to automate his manual test scripts. But it may also be the owner of the application who wants to have an automated test set for regression. In other words, just as there are different services, there are different clients.

Besides the client, the objective of a permanent test organisation may also determine the test services. The objective (often translated to a mission) reflects the ambition level and depends on many factors. For instance, the organisation may have been given an assignment from outside (“ensure that all test environments are set up and maintained efficiently” or “offer a solution to all issues surrounding test automation”). But an organisation may also create its own objective (“we execute all regression tests”). The objective of an organisation may change over time, as may the test service offering.

The permanent test organisation must be set up in such a way that it can offer optimal service (one or more services). The organisation is set by up by defining the tasks, authorisations and responsibilities in jobs for the employees. In addition processes will be defi ned, set up and selected for an organisation in which everything functions optimally. There are no hard and fast rules for this, nor is there such thing as the ideal setup.

BENEFITS, CONDITIONS AND POINTS OF CONCERN

Benefits

Usually the reasons to opt for a permanent test organisation are a reduction of lead time, cost reduction, and improvement of the quality of the test process. The idea is to organise specific elements of the test process across all projects, contrary to a project-based approach. But what exactly are the benefits of a permanent test organisation?

Optimal leveraging of (scarce) expertise

Often, test expertise is scarce within an organisation. Not only expertise in relation to setting up and executing test scripts, but also, for instance, to the deployment of test tools or setup of test environments. By combining all available expertise and consolidating all requests for this expertise into one place, you can optimise several issues. For instance, it is easier to find the required expertise because there is insight into all the available expertise. Furthermore, the available expertise can be divided across the various issues more effectively. Other requests can then be considered at less busy times. For instance, employees with knowledge of test tools can work for several projects at the same time.

Predictable quality of products

The services offered by the test organisation are standardised. There are products and processes with applicable standards. As such, there is no such thing as a new service - it has been done before and each employee knows how to execute it. Naturally every assignment is different, but the variation in services is limited to a minimum. This means that the quality of a service can be predicted. E.g., the service “setting up a test environment” can become a routine-based task because the process with the associated templates, tools, techniques and checklists can be used off the shelf. And because it is executed by experienced and qualified employees.

Short start-up time

Since the services are standardised, the start-up time is limited to a minimum. There is no need for courses to be attended, aspects to be tried out first, or the best approach to be determined. The start-up time of a service like setting up a test management tool can be limited to the minimum. Everything is readily available. It is just a matter of determining the client’s specific wishes and requirements, configuring them and starting up.

Continuous improvement of the process embedded in the organisation

The test organisation is responsible for the services provided and executes them. It can be decided to have each execution end with an assessment of the delivered service. The lesson learned from the assessment is fed back into the organisation and incorporated into the (new version of the) service. Such formal assessment and processing of results must be embedded into the processes of the test organisation.

Consolidation and development of experiences

Combining all of the available expertise with the above, where experiences are assessed and processed, results in a continuous learning effect. This is further strengthened by cross-fertilisation between employees.

Costs and lead time easier to plan

As earlier stated, the services are standardised, the employees have the expertise and experience to execute the services. “Reinventing the wheel” is unnecessary and there is clear insight into costs and results, and how long it takes.

Cost reduction due to centralisation and scale

Testing in different places in the organisation often means that negotiations with external suppliers occur in different places. By centralising all testers and test-related activities, the test organisation can make one central price agreement with the various suppliers, e.g. suppliers of tools and testers. Furthermore, the associated advantage of scale strengthens the negotiation position, and the possibility of getting a discount.

Example

A logistics service provider uses test tools for automated testing in various locations in the organisation. Every department purchased its own licences of the test tool, handled its own negotiations about the price with the various suppliers, and organised its own training for the tools used. When it was decided to centralise the setup and maintenance of the automated tests in the organisation, one party that used the test tools was created. This benefited not only the logistic service provider, but also the various suppliers. Because instead of many different (unclear) points of contact, they also had one single point of contact. This translated to improved efficiency, reducing the number of tools in use (from 23 to 6). The licence and training costs were also reduced by 35%.

Conditions

Not every (IT) organisation can have a permanent test organisation. The organisation must comply with a series of conditions, which are:

• To achieve the benefits described above, there must be a minimum quantity of work for the permanent test organisation. This is often the case in somewhat bigger organisations.
• There must be a culture in which formal work agreements are possible. This is necessary to ensure that the permanent test organisation can reach concrete agreements with its clients.
• The organisation must deal with repeatable processes and projects. Only then can a test organisation offer standard services.
• The organisation must be able to handle and accept central organisations, such as a permanent test organisation. Central organisations can be perceived as a threat (e.g. due to their size). This must not be the case.

The permanent test organisation as a catalyst for professionalisation

Many IT organisations want to professionalise, but do not have the time or money. One option is to find a lever. A lever due to which professionalisation in one specific location in the IT process chain has a direct impact on multiple parts of that process chain. A permanent test organisation has the potential of acting as a lever. This is because it is at the point where the results of all earlier processes in the development chain come together in an assessable end result. The test organisation is like the spider in the web. As such, it is in an excellent position to facilitate the mutual alignment of the parties and at the same time define requirements for the quality of the process and output. Several parties can benefit by setting up a permanent test organisation.

For instance, the test organisation can identify unclear or missing requirements at an early stage and detect potential problems in the development or maintenance process. Furthermore the test organisation keeps the parties involved on their toes as to the required quality and output, meaning that the end product as described at an early stage is actually realised. The test organisation is ‘the conscience’ of the IT process chain. In practice, this means that the test organisation is closely involved in every step in the chain and has an advisory role.

Example

The following awareness grew within a government institution.

• Testing occurs in many different places in the organisation (in the IT department for different projects on the client side within various organisations).
• Everyone has their own test approach (structured or unstructured).
• The work pressure for testers varies enormously (high time pressure before a release, low time pressure after a release), which means that testers sometimes have nothing to do (the so-called ‘waiting for work’ situation).
• The turnover among testers is huge, a lot of acquired knowledge is lost.
• The government institution’s reputation often is damaged because public software does not function properly.

An order is issued to set up a permanent test organisation with the following two objectives:

• The use of a method and techniques that conform to market standards.
• Being an attractive employer. The objective of the permanent test organisation was to increase productivity. Less ‘waiting for work’ and lower turnover of testers. A 10% cost saving is realised one year after the permanent test organisation is established. This is realised in part by deploying testers more effectively, and in part because less knowledge is being lost and therefore less training required.

Points of concern for the coming year are:

• Further professionalisation of HRM.
• Further development of test services.